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Abstract-  Research  over  the  last  9  years  has  resulted  in  an 
effective  mine  classification  approach  that  involves  the  use  of 
image-segmentation  based  screening  methods  followed  by 
multilayer  perceptron  networks  for  mine  classification.  The 
present  approach  centers  around  a  baseline  23  Feature  set  related 
to  highlight,  shadow,  and  highlight/shadow  contrast  statistic 
based  segmentations,  and  the  use  of  associated  statistical  and 
shape  related  factors.  In  the  work  described  here  we  investigate 
the  improvement  of  baseline  performance  by  incorporating  image 
texture  related  features  such  as  Cooccurrence  Matrix  related 
factors. 

I.  INTRODUCTION 

The  need  for  automated  processing  in  the 
detection/classification  of  mines  to  reduce  the  workload  of 
human  sonar  operators  is  clear.  The  mine- 
detection/classification  system  described  here  is 
partitioned  into  an  anomaly  screening  stage,  followed  by  a 
classification  stage  involving  the  calculation  of  features  on 
blobs,  and  their  input  into  a  multilayer  perceptron  neural 
network. 

In  order  to  obtain  sufficiently  low  false  alarm  densities 
for  an  actual  system,  several  strategies  can  be  employed. 
Multiple  mine  hunting  algorithms  may  be  fused,  i.e. 
detections/classifications  may  be  effectively  combined  by 
imposing  the  constraint  that  a  given  mine  like  blob  derived 
from  one  algorithm  must  intersect  at  least  one  blob 
obtained  from  each  of  an  aggregate  collection  of  distinct 
algorithms.  This  strategy  is  currently  employed  by  the 
Naval  Coastal  Systems  Station  (CSS)  in  combining  the 
results  obtained  from  the  CSS,  Alphatech,  Raytheon,  and 
Lockheed-Martin  mine  hunting  algorithms. 

To  improve  the  performance  of  individual  Mine 
Classification  algorithms,  new  feature  sets  are  continually 
under  investigation,  particularly  as  the  resolution  and 
quality  of  available  side-scan  sonar  data  improves. 

In  the  discussion  that  follows  section  2.0  defines  the 
overall  mine  detection/classification(CAD/CAC)  algorithm 
structure.  Defining  our  CAD/CAC  algorithm  will  involve 
defining  the  formation  of  our  “Baseline”,  23  dimensional 
feature  set,  and  additional  features  considered  here  for 


augmenting  the  Baseline  set,  consisting  of  Haralick's 
Cooccurrence  Matrix  derived  factors[l]. 

In  section  3.0  classification  performance  results 
obtained  from  fusing  classifiers  based  on  the  baseline  and 
texture  features,  respectively,  are  compared  with  baseline 
performance.  Finally,  in  section  5.0  conclusions  are  made. 

II.  MINE  HUNTING  ALGORITHM 
STRUCTURE 

The  structure  of  the  aggregate  detection/classification 
algorithm  studied  here  is  summarized  in  Fig.  1.  The  image 
normalization  algorithm[2]  in  Fig.  1  was  developed  by 
Dr.Gerald  Dobeck  of  the  NSWC  CSS(Naval  Surface 
Warfare  Center  Coastal  System  Station),  as  well  as  others, 
and  consists  in  the  use  of  a  combined  forward/backward 
low-pass  filtering  approach  to  estimate  local  background 
intensity  at  each  pixel  along  a  range  profile.  The  input 
image  intensity  along  the  range  profile  is  then  normalized 
by  those  local  background  intensity  estimates.  This  results 
in  more  homogeneous  background  characteristics,  making 
highlights  and  shadows  more  uniformly  visible.  The 
purpose  of  the  anomaly-screening  algorithm  is  to  extract 
blobs;  i.e.  collections  of  connected  pixels,  which 
correspond  to  candidate  mine-like  (ML)  objects.  These 
candidate  ML  blobs  will  be  referred  to  as  image  Tokens. 

A  vector  of  classification  features,  to  be  defined  in  the 
discussion  that  follows,  is  then  calculated  for  each  image 
Token.  A  trained  multilayer  perceptron  network  is  then 
employed,  using  the  Token  classification  feature  vectors  as 
inputs,  in  order  to  obtain  “target”  and  “clutter”  output 
statistics,  which  may  be  viewed  as  approximate  posterior 
probabilities.  The  logarithm  of  the  ratio  of  “target”  over 
“clutter”  output  node  statistics  is  calculated  and 
corresponds  to  a  log-likelihood  ratio  statistic-  LLR.  These 
LLR  values  are  ranked  and  thresholded  to  determine  a 
final  output  ML  Token  list. 
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Fig.  1  Aggregate  Detection/Classification  Algorithm  Structure 

The  above  discussion  defines  the  overall  structure  of 
the  mine-detection/classification  algorithm.  Next,  we 
describe  in  more  detail  the  generic  anomaly-screening 
algorithm  depicted  in  Fig.  2,  which  is  employed  as  a 
building  block  in  the  actually  employed  anomaly¬ 
screening  algorithm.  In  general,  an  anomaly  statistic  is 
selected  as  some  measure  quantifying  the  deviation  of 
behavior  over  some  local  image  neighborhood,  as 
compared  to  local-background  characteristics.  The 
Markov  Random  Field(MRF)  based  anomaly  statistic[3] 
conceived  in  past  work  measured  the  degree  of 
anomalousness  of  a  specified  neighborhood  of  pixels 
associated  with  a  row  in  the  least  squares  data  matrix  for 
fitting  a  first  order  MRF  model.  In  the  current  image 
databases  under  investigation,  ML  targets  have  relatively 
small  highlight  regions,  followed  by  longer  shadows. 
Hence,  in  the  present  case  a  highlight/shadow  contrast 
based  anomaly  statistic  tHs  is  formed  as 

tHS  =  (mH-BNA  )/(HNA-BNA ) 

+  ( mS- BN  A )/( SNA  -  BN  A )  (1) 

,  where  mH,  mS  denote  intensity  means  over  adjoining  ky 
by  kxh,  ky  by  kxs  windows,  where  ky  denotes  a  pixel  length 
in  the  along  track  direction,  and  (kxh,  kxs)  denotes  pixel 
lengths  in  the  range  direction  for  highlight  and  shadow, 
respectively.  The  parameters  IINA ,  BNA,  SNA  denote 
average  highlight,  background,  and  shadow  intensities  in 
the  normalized  image,  respectively.  Note,  in  addition,  that 
kxs  is  allowed  to  vary  linearly  as  a  function  of  range. 

Given  the  selection  of  an  anomaly  statistic,  an  estimate  for 
the  anomaly  statistic  cumulative  distribution  function. 


Fcda(. ) ,  is  computed,  and  pixels  are  extracted  by  requiring 
that 

Fcda(ta)  >  1.0-  £  *01,  (2) 

where  ta  denotes  the  anomaly  statistic  value  defined  at  a 
given  pixel,  and  £  denotes  the  top  percentage  of  pixels  to 
be  extracted.  Next,  standard  computer  vision[4]  based 
region  labeling  algorithms  are  employed  to  obtain 
disconnected  image  Tokens  or  blobs.  The  final  stages  of 
the  anomaly-screening  algorithm  involve  the  filtering  of 
blobs  to  identify  the  most  likely  ML  blob  candidates.  The 
maximum  anomaly  statistic  and  pixel  count  associated 
with  each  blob  are  computed  and  denoted  by  MP,  PC, 
respectively.  The  associated  ranks  rMP,  rPC  are  computed 
and  the  aggregate  rank  statistic  rMP_PC  is  defined  by 

rMP_PC  =  min(rMP,rPC).  (3) 

Finally,  the  image  Tokens  returned  by  the  screening 
algorithm  are  those  for  which 

PCmin  <  PC  <PCmax  (4) 

rMP_PC  <=  Thresh_screener  (5) 


,where  Thresh_screener  denotes  the  rank  threshold 
required  for  Token  selection.  In  the  discussion  that  follows 
we  will  refer  to  the  anomaly- screening  algorithm  as 
retaining  the  top  Thresh_screener  Tokens. 


Fig.  2  Generic  Anomaly-Screening  Algorithm  Structure 
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Given  the  above  description  of  a  generic  anomaly¬ 
screening  algorithm,  we  can  define  the  actual  screening 
algorithm  employed.  In  our  case,  the  generic  anomaly 
screening  algorithm  defined  above  is  actually  applied  for 
three  distinct  anomaly  statistics,  and  the  results  are 
combined  in  a  fashion  described  in  the  discussion  that 
follows.  We  will  use  the  notation  A (7,  b)  to  denote  the 
anomaly- screening  algorithm  employing  statistic  t,  and 
“filtering”  parameter  vector-  b,  where  for  example. . . 

bi  =  PCmax  (6) 

b2  =  PCmin 

b<  =  Thresh_screener 

b4=  £ 

and  the  above  parameters  are  as  defined  in  (2),  (4),  and  (5). 
In  our  case  we  make  use  of  a  highlight/shadow  contrast 
statistic  tHs,  as  defined  by  (1),  and  an  appropriate  set  of 
screening  parameters-  bus.  In  addition,  we  make  use  of 
what  will  be  termed  “highlight”,  tH,  and  “shadow”,  ts, 
anomaly  statistics  together  with  associated  screening 
parameter  vectors  bH,bs,  respectively. 

Letting  S(t,  b)  denote  the  image  Tokens  resulting  from 
applying  screening  algorithm  A  (7,  b),  then  the  aggregate 
screening  algorithm  we  employ  corresponds  to  extracting 
the  collection  of  “filtered”  Tokens  SF,  which  are  the  subset 
of  S(tHS,  bus),  satisfying  the  below  constraint  for  each 
Token  T  in  SF 

(T  has  a  non-null  intersection  with  S(tH,  bH))  or 

(T  has  a  non-null  intersection  with  S(ts,  bs))  (7) 

The  motivation  behind  imposing  the  constraint  in  (7)  is 
that  Situs,  />//<;j-Tokcns,  when  they  correspond  to  genuine 
mine -like  objects,  should  be  associated  with  either  a  local 
highlight  or  local  shadow  region. 

In  our  case,  the  highlight  detection  statistic  tu(i,j)  is 
roughly  defined  as 

tH(i,j)  =  n(i,j)/nM  (8) 

,  where  n(i,j)  is  computed  as  an  average  of  squared 
deviations  from  a  local  normalized  image  median,  in  the  8 
directions  corresponding  to  the  nearest  neighbors  at  a 
given  pixel,  and  nM  corresponds  to  a  median  value  for  n(.,.) 
in  a  local  rectangular  ring  centered  at  (i,j).  The  form 
adopted  for  tH(i,j)  can  be  motivated  as  a  simplified  form  of 
the  MRF  based  anomaly  statistic  mentioned  above. 

Finally,  note  that  the  adopted  “shadow”  detection 
statistic,  ts(i,j),  is  defined  as 

ts(ij)  =  -Mi,j)  (9) 


where  1N( i,j )  denotes  the  normalized  image  intensity. 
Hence,  “shadow”  blobs  are  derived  by  selecting  the  “tail” 
of  the  ts  distribution. 

The  above  completes  our  discussion  of  the  aggregate 
anomaly  screening  algorithm  employed  in  the  context  of 
Figure- 1.  Next,  we  complete  our  discussion  of  the  mine¬ 
hunting  algorithm  structure  by  defining  the  feature  vector 
employed  in  the  classification  stage  of  the  algorithm.  In 
defining  the  calculated  classification  features  it  is 
important  to  note  that  three  distinct  segmentations  are 
involved,  corresponding  to  the  use  of  the  tHS,  tH,  ts 
“anomaly”  statistics,  respectively,  and  that  in  addition  the 
normalized  image-  IN-  is  also  involved.  Hence,  in 
describing  the  features  it  is  necessary  to  denote  which 
segmentation  is  involved,  and  the  underlying  image 
statistic  employed  in  any  calculations.  The  designations 
HS,  H,  S  will  be  employed  to  denote  segmentations 
derived  from  the  tHS,  tu,  ts  “anomaly”  statistics, 

respectively.  The  current  baseline  feature  vector,  J  ,  is 

B 

23-dimensional  and  defined  as  follows: 

fi  =  PC  for  HS-segmentation  (10) 

f2=  mini rPC,  rMP)  for  HS-segmentation  (11) 

fi  =  “local”  summation  of  unfiltered  blob  PC  for  HS- 
segmentation  (12) 

f4  =  mean  tHS  over  blob  for  HS-segmentation  (13) 

fi  =  standard  deviation  oftHS  over  blob  for  HS- 
segmentation  (14) 

f6  =  “local”  summation  of  unfiltered  blob  count  for  HS- 
segmentation  (15) 

fi  =  MP  for  tHS  over  blob  for  HS-segmentation  (16) 

fs  =  (1,0)  valued  indicator  for  existence  of  intersecting  H- 
segmentation  blob  (17) 

fi  =  MP  for  tHover  intersecting  H-segmentation  blob(18 ) 
fio  =  PC  for  intersecting  H-segmentation  blob  (19) 

fii  =  (1,0)  valued  indicator  for  existence  of  intersecting  S- 
segmentation  blob  (20) 

fi2  =  PC  for  intersecting  S-segmentation  blob  (21) 

fis  =  mean  IN  over  blob  for  HS-segmentation  (22) 

f14  =  maximum  IN  over  blob  for  HS-segmentation  (23) 

f  1$  =  standard  deviation  offi  over  blob  for  HS- 
segmentation  (24) 

fi6  =  skewness  coefficient  ofIN  over  blob  for  HS- 
segmentation  (25) 

fi7  =  kurtosis  coefficient  offi  over  blob  for  HS- 
segmentation  (26) 

fis  =  perimeter  of  blob  for  HS-segmentation  (27) 

fig  =  (16*PC)/(perimeter*perimeter)  for  HS-segmentation 
blob  (28) 

fio  =  perimeter/(2*(bounding-box-width  +  bounding-box- 
height))  for  HS-segmentation  blob  (29) 

fii  =  (major-axis  -  minor-axis )/  (major-axis  +  minor-axis) 
for  HS-segmentation  blob  (30) 

f22  =  major-axis  (31) 

fis  =  orientation  of  HS-segmentation  blob  (32) 
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The  features  (f,t,  f$ )  are  defined  by  summations  over  all 
unfiltered  blobs  intersecting  a  50x50  window  centered  at 
the  HS -segmentation  blob-centroid.  The  features  (f2j}  f'22, 
f2s)  are  computed  by  fitting  an  ellipsoid  to  the  HS- 
segmentation  blob  using  computed  “central”  x,  y,  and  xy 
moment  information. 


The  above  discussion  defined  the  baseline  23  feature 

set,  /  .  In  the  course  of  the  work  described  here  we 

J  B 

define  a  Cooccurrence  related  feature  vector,  /  ,  a 

J  cooc 

selected  subset  of  which  is  employed  as  input  to  a  second 
MLP  classifier,  whose  output  is  then  employed  in 
combination  with  the  output  of  the  baseline  classifier,  in 
order  to  obtain  improved  Mine  Classification  performance. 
Here,  Cooccurrence  matrices  will  be  conceived  as  two- 
dimensional  discrete  distributions, 

PhVJ).  PyV’j)’  PRD O'.j),  Pld O'JX  where 

the  subscripts  H,  V,  RD,  LD  denote  distributions  associated 
with  pairs  of  pixels  separated  by  a  specified  number  of 
shifts  in  the  horizontal,  vertical,  right-diagonal,  and  left- 
diagonal  directions,  respectively.  Note,  that  i,  j  denote 


discrete  quantities  assuming  values  0...  Ng  —  1,  obtained 

by  quantizing  some  image  derived  from  the  underlying 
side-scan  data.  The  calculation  of  a  26-dimensional 
Coocurrence  Matrix  related  feature  vector,  denoted  here 

by,  /  ,  is  described  as  follows,  where  s-denotes  the 

J  COOC.s 

specified  number  of  shifts  employed  in  the  definition  of 
the  Cooccurrence  distributions: 

1)  Estimate  cumulative  histogram  for  window  grey- 

levels  to  define  Ng  equal  probability 
quantization  levels 

2)  Calculate  Empirical  Cooccurrence  distributions 


3) 


4) 


Pji.j).  pr«.j\  pji.j).  pjij ) 

Calculate  Haralick’s  13 -dimensional 


Cooccurrence  Related  Feature  Vector, 


/ 


COH 


with  components  f  ■  ■  ■  /  .  for  each 

J  COHA  J  COH,  13 

of  the  four  directions,  i.e.  horizontal,  vertical, 
right-diagonal,  and  left-diagonal. 

Calculate  feature  averages  over  direction. 


/ 


ACOH,  1 


/ 


ACOH,  13’ 


and  the  ranges  of 


feature  variation  over  direction 


•/, 


/, 


ACOH , l 

,  in  order  to  define  an  aggregate  26 


ACOH, 13 

dimensional  feature  vector,  denoted  by 

/ 


COOC,s 


Given  the  above  discussion,  for  a  sequence  of  shift 
magnitudes  ^  . . .  g ,  the  aggregate  Cooccurrence 

Matrix  related  feature  vector,  /  ,  is  defined  as 

J  cooc 


having  partitions . . . 


/ 


cooc,sl 


cooc,s 


.  In 


our  case  we  selected  q=3,  ^  =1,  ^  =2,  =3.  In 

addition,  we  let  Ng  =8,  the  number  of  bins 

employed  in  empirical  cumulative  histogram 
calculation  was  15,  and  the  image  window  dimension 
employed  was  10.  Finally,  note  that  two  distinct  image 
statistics  where  investigated  for  the  purpose  of 
discretization  in  order  to  calculate  Cooccurrence 
Matrix  distributions.  In  each  case  the  window  was 
centered  at  the  //.S'-segmentation  blob  centroid,  but  in 
one  case(A),  the  //.S-stati  Stic  was  employed  to  define 
image  intensities,  while  in  the  second  case(B),  the 
normalized  side-scan  image  was  employed  to  define 
image  intensities. 


The  above  serves  to  define  the  overall  framework 
for  calculation  of  a  Cooccurrence  Matrix  related 

feature  vector,  /  .  The  discussion  that  follows 

J  cooc 

defines  the  calculation  of  Haralick's  13-dimensional 

feature  vector,  /  .  The  notation  p(i,  j)  will  be 

employed  to  denote  any  of  the  directional 
Cooccurrence  matrices  mentioned  above.  In  addition, 
the  following  notations  will  simplify  the  construction 
of  the  features: 


p  (0=  ^p(hj) 


7=0 


AG-1 

p  (;)= 


i= 0 


Ns 


!=  0 


N,-' 

n  TjpU) 


7=0 


NC  2 

CT  1 0-/0  PS o 


;= o 


(33) 

(34) 

(35) 

(36) 

(37) 
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cr2,=  SO'-^y)2  p(j) 


j= 0 


AT1 


Px+y«)  =  \Lp(.Uk-i)[0<k<Ng- 1 


(38) 


(39) 


Z  pa*-o[wg-i^2(#g-i) 

i=k~N  +1 


Pu-/k)  = 


Nr' 

(  )[k  =  0} 


i=0 


/  N  g-l~k  N  g~l  \ 

Z  po\i  +  *)+  Z  p(i,i  -  k)  | 

V  1=0  /=£  / 


[i<^<AA  -ij 


(40) 


f  =(<T2  +  <T2)/2  (48) 

J  coha  ^  x  ^  y 

fcoH,r  Z  Zrftiva+O'-/)*  (49> 

;=0  y'=0 


f  =  Z  *  p  (o  ^50) 

7  COH  ,6  r  x+y 

1=0 


^Af,-1)  .  2 

/COff.7=  Z  (i-f  COH, 6~)  Px+y®  (51) 

1=0 


2WS-1} 


1=0 


(52) 


Nr1 

HX=Z  -p(i)Hpxd)) 


i=0 


Nr' 

hy=Y,  -puwpm 


7=0 


(41) 


(42) 


A^r'A^r1 

//ATI  =  Z  Z  -  P(k  j )  ln( p  O')  p  (/'))  (43) 


i=0  7=0 


AT1  AT1 

HXY2  =  Z  Z  -  Pt  (0  P  x  U)  ln( p  (0  p  O'))  (44> 


1=0  7=0 

Given  the  above  definitions,  the  13  components  of 

Haralick's  feature  vector,  f  ,  can  be  defined  as 
7  COH 

follows: 

A'  ;A:  1 

/«,*.,=  Z  Zp0,7) 

1=0  7=0 

AT,-1  AT,-1  2 

/c«.i=  z  Z  0‘— 7)  *‘'7> 

1=0  7=0 


(45) 


(46) 


Mr1  Air1 

/cow.3  =  (  z  Z Mhj)-pu  Ad  d)  (47) 


AA-'AA-1 

/coh,9=  Z  Z-P0o‘)ln(p0o'))  (53) 

i=0  7=0 

/=  Variance  of  T)  (k)  distribution  (54) 

COH  .10  r\x-y\ 


N  s 


L,=  Z-  >n(p  (*)) 

&=0  1/1  1/1 


(55) 


i=0  7=0 


f  =  (  f  -  HXYl )  /  ma x(//X ,  //7)  (56) 

COH  ,\2  J  COH, 9 

f„u- (l-exp(-2(//ZF2-/COH.9)))’  (57) 


III.COMPARISON  OF  TEXTURE  MEASURE 
AUGMENTED,  AND  BASEUINE  CUASSIFIER 
PERFORMANCE  RESUUTS 

Recall  in  the  above  discussion  that  two  cases, 
designated  (A),  (B)  are  defined  corresponding  to  the  use  of 
HS-statistic  data,  and  normalized  side-scan  image  data, 
respectively,  in  constructing  the  aggregate  78-dimensional 

Cooccurrence  related  feature  vector,  denoted  by  f 

7  cooc 

In  each  case,  the  Genetic  Algorithm  based  approach  to 
feature  selection  embedded  in  Neuralware’s  state  of  the  art 
Cascade  Correlation  Network  Classification  model  builder 
is  employed  to  select  18,  and  10  features  for  cases  (A), 

(B),  respectively. 
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In  Fig. -3  the  Baseline  23F  classification  performance  is 
compared  with  the  18F  and  10F  performance, 
respectively.  Note,  that  FAI  denotes  the  average  number  of 
false  alarms  per  image,  and  PCD  denotes  the  cumulative 
Mine  classification  probability,  over  a  test  data  set  of  244 
images.  This  test  data  set  is  distinct  from  the  56  images 
employed  for  training.  In  addition,  in  Fig. -3  upper  and 
lower  90%  coverage  probability  confidence  bounds  for  the 
23F  baseline  performance  are  displayed,  as  obtained  using 
the  Boostrap  Based  approach  detailed  in  [5],  These  results 
demonstrate  that  the  10F,  18F  cases  have  classification 
performance  that  is  significantly  worse,  in  a  statistical 
sense,  than  the  23F  baseline  case. 
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In  order  to  investigate  the  performance  achievable  by 
the  fusion  of  the  baseline  23F  and  either  of  Cooccurrence 
Matrix  Feature  cases  (A),  (B),  we  consider  the  selection  of 
joint/simultaneous  classification  thresholds  associated  with 
the  output  of  a  23F  classifier,  and  a  Cooccurrence  Matrix 
Feature  related  classifier,  respectively,  in  order  to 
minimize  the  achieved  number  of  false  alarms,  at  each 
specified  number  of  classified  targets.  In  Fig.  4  and  5  we 
present  the  (FAI,  PCD)  curves  associated  with  the  fused 
(23F,  18F)  and  (23F,  10F),  cases  respectively,  together 
with  the  baseline  23F  performance.  In  addition,  in  each 
case  we  present  upper  and  lower,  90%  coverage  bounds 
associated  with  the  fused  performance.  In  this  case 
however,  the  resulting  Bootstrap  derived  bounds  are 
conditioned  on  a  specified  range  of  classified  targets  being 
achievable,  based  on  appropriate  joint  threshold  selections 
associated  with  the  outputs  of  individual  classifiers. 

Hence,  this  is  equivalent  to  conditioning  on  a  fixed 
collection  of  PCD  values  being  achievable,  and 
determining  upper  and  lower  bounds  on  the  achievable 
FAI  performance. 
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Fig.-3  Comparison  of  Classification  Performance  for 
Baseline  23F  and  Cooccurrence  Matrix  Feature  Derived 
Cases  (A),  (B),  respectively 


Fig.-4  Comparison  of  Classification  Performance  for 
Baseline  23F  and  Fused  (23F,18F)  Cases,  Respectively 
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Fig.-5  Comparison  of  Classification  Performance  for 
Baseline  23F  and  Fused  (23F,10F)  Cases,  Respectively 

IV.CONCLUSIONS 

Results  in  Fig.  3-5  demonstrate  that  while  the 
Cooccurrence  related  features  computed  from  the 
normalized  side-scan  image  are  more  powerful  for 
mine/clutter  discrimination,  than  those  computed  from  the 
HS-segmentation  statistic,  the  fusing  of  either  set  of 
features  with  the  Baseline  23F  set  does  not  result  in 
statistically  significant  performance  improvement,  relative 
to  the  Baseline  case. 
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